A unifying probabilistic perspective for spectral dimensionality reduction: Insights and new models

We introduce a new perspective on spectral dimensionality reduction which views these methods as Gaussian Markov random fields (GRFs). Our unifying perspective is based on the maximum entropy principle which is in turn inspired by maximum variance unfolding. The resulting model, which we call maximum entropy unfolding (MEU) is a nonlinear generalization of principal component analysis. We relate the model to Laplacian eigenmaps and isomap. We show that parameter fitting in the locally linear embedding (LLE) is approximate maximum likelihood MEU. We introduce a variant of LLE that performs maximum likelihood exactly: Acyclic LLE (ALLE). We show that MEU and ALLE are competitive with the leading spectral approaches on a robot navigation visualization and a human motion capture data set. Finally the maximum likelihood perspective allows us to introduce a new approach to dimensionality reduction based on L1 regularization of the Gaussian random field via the graphical lasso

Bottom-up data Trusts: Disturbing the ‘one size fits all’ approach to data governance

Key Points The current lack of legal mechanisms that may plausibly empower us, data subjects to ‘take the reins’ of our personal data leaves us vulnerable. Recent regulatory endeavours to curb contractual freedom acknowledge this vulnerability but cannot, by themselves, remedy it—nor can data ownership. The latter is both unlikely and inadequate as an answer to the problems at stake. We argue that the power that stems from aggregated data should be returned to individuals through the legal mechanism of Trusts. Bound by a fiduciary obligation of undivided loyalty, the data trustees would exercise the data rights conferred by the GDPR (or other topdown regulation) on behalf of the Trust’s beneficiaries. The data trustees would hence be placed in a position where they can negotiate data use in conformity with the Trust’s terms, thus introducing an independent intermediary between data subjects and data collectors. Unlike the current ‘one size fits all’ approach to data governance, there should be a plurality of Trusts, allowing data subjects to choose a Trust that reflects their aspirations, and to switch Trusts when needed. Data Trusts may arise out of publicly or privately funded initiatives. By potentially facilitating access to ‘pre-authorized’, aggregated data (consent would be negotiated on a collective basis), our data Trust proposal may remove key obstacles to the realization of the potential underlying large datasets

Computationally Efficient Convolved Multiple Output Gaussian Processes

Recently there has been an increasing interest in methods that deal with multiple outputs. This has been motivated partly by frameworks like multitask learning, multisensor networks or structured output data. From a Gaussian processes perspective, the problem reduces to specifying an appropriate covariance function that, whilst being positive semi-definite, captures the dependencies between all the data points and across all the outputs. One approach to account for non-trivial correlations between outputs employs convolution processes. Under a latent function interpretation of the convolution transform we establish dependencies between output variables. The main drawbacks of this approach are the associated computational and storage demands. In this paper we address these issues. We present different sparse approximations for dependent output Gaussian processes constructed through the convolution formalism. We exploit the conditional independencies present naturally in the model. This leads to a form of the covariance similar in spirit to the so called PITC and FITC approximations for a single output. We show experimental results with synthetic and real data, in particular, we show results in pollution prediction, school exams score prediction and gene expression data

Gaussian Process Latent Force Models for Learning and Stochastic Control of Physical Systems

© 1963-2012 IEEE. This paper is concerned with learning and stochastic control in physical systems that contain unknown input signals. These unknown signals are modeled as Gaussian processes (GP) with certain parameterized covariance structures. The resulting latent force models can be seen as hybrid models that contain a first-principle physical model part and a nonparametric GP model part. We briefly review the statistical inference and learning methods for this kind of models, introduce stochastic control methodology for these models, and provide new theoretical observability and controllability results for them.The work of S. Sarkka was financially supported by the Academy of Finland. The work of M. A. Alvarez was supported in part by the EPSRC under Research Project EP/N014162/1

Variational inference for latent variables and uncertain inputs in Gaussian processes

The Gaussian process latent variable model (GP-LVM) provides a flexible approach for non-linear dimensionality reduction that has been widely applied. However, the current approach for training GP-LVMs is based on maximum likelihood, where the latent projection variables are maximised over rather than integrated out. In this paper we present a Bayesian method for training GP-LVMs by introducing a non-standard variational inference framework that allows to approximately integrate out the latent variables and subsequently train a GP-LVM by maximising an analytic lower bound on the exact marginal likelihood. We apply this method for learning a GP-LVM from i.i.d. observations and for learning non-linear dynamical systems where the observations are temporally correlated. We show that a benefit of the variational Bayesian procedure is its robustness to overfitting and its ability to automatically select the dimensionality of the non-linear latent space. The resulting framework is generic, flexible and easy to extend for other purposes, such as Gaussian process regression with uncertain or partially missing inputs. We demonstrate our method on synthetic data and standard machine learning benchmarks, as well as challenging real world datasets, including high resolution video data.This research was partially funded by the European research project EU FP7-ICT (Project Ref 612139 \WYSIWYD"), the Greek State Scholarships Foundation (IKY) and the University of She eld Moody endowment fund. We also thank Colin Litster and \Fit Fur Life" for allowing us to use their video les as datasets

Overlapping Mixtures of Gaussian Processes for the data association problem

In this work we introduce a mixture of GPs to address the data association problem, i.e. to label a group of observations according to the sources that generated them. Unlike several previously proposed GP mixtures, the novel mixture has the distinct characteristic of using no gating function to determine the association of samples and mixture components. Instead, all the GPs in the mixture are global and samples are clustered following "trajectories" across input space. We use a non-standard variational Bayesian algorithm to efficiently recover sample labels and learn the hyperparameters. We show how multi-object tracking problems can be disambiguated and also explore the characteristics of the model in traditional regression settings

Recurrent Gaussian processes

We define Recurrent Gaussian Processes (RGP) models, a general family of Bayesian nonparametric models with recurrent GP priors which are able to learn dynamical patterns from sequential data. Similar to Recurrent Neural Networks (RNNs), RGPs can have different formulations for their internal states, distinct inference methods and be extended with deep structures. In such context, we propose a novel deep RGP model whose autoregressive states are latent, thereby performing representation and dynamical learning simultaneously. To fully exploit the Bayesian nature of the RGP model we develop the Recurrent Variational Bayes (REVARB) framework, which enables efficient inference and strong regularization through coherent propagation of uncertainty across the RGP layers and states. We also introduce a RGP extension where variational parameters are greatly reduced by being reparametrized through RNN-based sequential recognition models. We apply our model to the tasks of nonlinear system identification and human motion modeling. The promising obtained results indicate that our RGP model maintains its highly flexibility while being able to avoid overfitting and being applicable even when larger datasets are not available

Validation of nonlinear PCA

Linear principal component analysis (PCA) can be extended to a nonlinear PCA by using artificial neural networks. But the benefit of curved components requires a careful control of the model complexity. Moreover, standard techniques for model selection, including cross-validation and more generally the use of an independent test set, fail when applied to nonlinear PCA because of its inherent unsupervised characteristics. This paper presents a new approach for validating the complexity of nonlinear PCA models by using the error in missing data estimation as a criterion for model selection. It is motivated by the idea that only the model of optimal complexity is able to predict missing values with the highest accuracy. While standard test set validation usually favours over-fitted nonlinear PCA models, the proposed model validation approach correctly selects the optimal model complexity.Comment: 12 pages, 5 figure

Experimental GHZ Entanglement beyond Qubits

The Greenberger-Horne-Zeilinger (GHZ) argument provides an all-or-nothing contradiction between quantum mechanics and local-realistic theories. In its original formulation, GHZ investigated three and four particles entangled in two dimensions only. Very recently, higher dimensional contradictions especially in three dimensions and three particles have been discovered but it has remained unclear how to produce such states. In this article we experimentally show how to generate a three-dimensional GHZ state from two-photon orbital-angular-momentum entanglement. The first suggestion for a setup which generates three-dimensional GHZ entanglement from these entangled pairs came from using the computer algorithm Melvin. The procedure employs novel concepts significantly beyond the qubit case. Our experiment opens up the possibility of a truly high-dimensional test of the GHZ-contradiction which, interestingly, employs non-Hermitian operators.Comment: 6+6 pages, 8 figure

Mindfulness based interventions in multiple sclerosis: a systematic review

<b>Background</b> Multiple sclerosis (MS) is a stressful condition; depression, anxiety, pain and fatigue are all common problems. Mindfulness based interventions (MBIs) mitigate stress and prevent relapse in depression and are increasingly being used in healthcare. However, there are currently no systematic reviews of MBIs in people with MS. This review aims to evaluate the effectiveness of MBIs in people with MS.<p></p> <b>Methods</b> Systematic searches were carried out in seven major databases, using both subject headings and key words. Papers were screened, data extracted, quality appraised, and analysed by two reviewers independently, using predefined criteria. Study quality was assessed using the Cochrane Collaboration risk of bias tool. Perceived stress was the primary outcome. Secondary outcomes include mental health, physical health, quality of life, and health service utilisation. Statistical meta-analysis was not possible. Disagreements were adjudicated by a third party reviewer.<p></p> <b>Results</b> Three studies (n = 183 participants) were included in the final analysis. The studies were undertaken in Wales (n = 16, randomised controlled trial - (RCT)), Switzerland (n = 150, RCT), and the United States (n = 17, controlled trial). 146 (80%) participants were female; mean age (SD) was 48.6 (9.4) years. Relapsing remitting MS was the main diagnostic category (n = 123, 67%); 43 (26%) had secondary progressive disease; and the remainder were unspecified. MBIs lasted 6–8 weeks; attrition rates were variable (5-43%); all employed pre- post- measures; two had longer follow up; one at 3, and one at 6 months. Socio-economic status of participants was not made explicit; health service utilisation and costs were not reported. No study reported on perceived stress. All studies reported quality of life (QOL), mental health (anxiety and depression), physical (fatigue, standing balance, pain), and psychosocial measures. Statistically significant beneficial effects relating to QOL, mental health, and selected physical health measures were sustained at 3- and 6- month follow up.<p></p> <b>Conclusion</b> From the limited data available, MBIs may benefit some MS patients in terms of QOL, mental health, and some physical health measures. Further studies are needed to clarify how MBIs might best serve the MS population.<p></p&gt